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Abstract: In this paper we present the way a multi-criteria decision making methodology is applied in 
the case of agent-based selection of offered learning objects. The problem of selection is modeled as a 
decision making one, with the decision variables being the learner model and the learning objects’ 
educational description. In this way, selection of educational content is based on dynamic data input 
collected at the time of the decision. This methodology is studied in the context of an agent-based e- 
market for educational content brokering, and is engaged by the broker agents recommending learning 
objects to learners, according to their cognitive style. 



Introduction 

In agent-mediated educational content brokering, artificial agents are responsible of locating offered 
learning objects, negotiating with the content providers on the terms of the service provision, and for presenting the 
learner with those offers that best match his/her needs and preferences. Successful completion of this task is based 
on two parameters: the capability of the mediating agent to understand and model the user needs, and its capability 
to evaluate all available learning objects and recommend the most suitable. The most common ways to introduce 
intelligent brokering for educational content are based on techniques from Artificial Intelligence (AI); knowledge 
representation, reasoning, and expert systems, often offer tools and techniques in the developers of such systems 
since they allow the agent to behave intelligently in two ways; first, with the ability to store the knowledge of the 
experts (for example using a knowledge base); second, using the previous knowledge to infer rational decisions. 

In the last years though, most agent-based e-commerce systems engage methodologies and tools which 
originally come from sciences other than AI; these include methods and techniques from Game Theory, Optimization 
and Decision Making (Guttman et al, 1998), In this paper, we are going to approach the problem of content selection 
(know as the recommendation problem in e-commerce systems) from a decision-making point of view. We are going 
to introduce how a multi-criteria decision making (MCDM) methodology is applied in the case of agent-based 
educational content brokering, and we are going to discuss the benefits and drawbacks of this approach. 

There is an interesting feature in this approach: it replaces searching knowledge inside a knowledge base, 
using constraints posed by logical expressions and logic rules, with an evaluation of knowledge using mathematical 
formulations. It can therefore provide measures of “how much better one decision is compared to another”, even if 
both decisions are rational and solve the constraints problem. This usually sounds rather awkward though - how can 
we represent the expert knowledge in terms of mathematical expressions? Logic variables (CategoryOfUser=’This’) 
and logic rules (IF CategoryOfUser=’This’ THEN TypeOfContent=’That’) are simple, and easily comprehended; how 
is the learner going to be modeled using mathematical variables, and how is the expert going to express the 
experience in a mathematical form? 

We will try to give answers to these questions by introducing the example of modeling the learners with the 
classic Honey and Mumford cognitive styles model (1992), We will introduce the basic principles of a multi-criteria 
decision making methodology, and study which parameters of the Honey & Mumford model serve as criteria for the 
selection of the available content (learning objects); we will also study a method that can be used to carry out this 
selection. Finally, we will present how this decision-making procedure is incorporated into an agent-based system, 
and how the broker agents operate it. 
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The Honey & Mumford Model 



Tennant (1988) defined cognitive style as ’‘an individuals characteristic and consistent approach to 
organizing and processing information’*. There are several learning and cognitive styles theories and models, which 
categorize learners in terms of instructional preferences, information processing and personality styles, and are 
usually employed for the realization of individualized instruction (for a survey, see Sampson et al, 2002). In order to 
demonstrate how a cognitive style model can be used in order to create the multi-criteria selection model, we will 
engage the classic Honey and Mumford model. The Honey and Mumford model (1992) is a cognitive style model, 
developed for use in commerce, management and training situations. It categorizes learners according to the 
following dimensions of a person’s learning style: Theorist, Activist, Reflector and Pragmatist. In order to rate 
learners on each one of the categories, the model uses the answers in a specially designed Learning Styles 
Questionnaire (LSQ) of 80 questions, with binary answers of “Correct” and “False”, and after an internal processing 
of the results, it provides a percentage weight for each category. Therefore, the learner’s style is defined depending 
on the weight at each learning style category. 
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Figure 1: An example learner’s model after a LSQ is processed. It provides a measure of how the learner ‘scored’ in 
every category (source: PSI-Press, http://www.psi-press.co.uk/) 

From a practical point of view, it is important that a learner’s model describes not only how learners are 
categorised, but also how the instruction method should be adapted for each learner category (Spector, 2001). The 
real complexity for the designers of e-leaming systems arises when they try to match subject matter with learner 
characteristics and appropriate instructional methods. Such a process includes both learner modeling (using the 
Honey & Mumford model results - see Figure 1) and description of the educational properties of the learning content 
(that is e.g. a description of how suitable a course is for the learners having a specific cognitive style). The matching 
procedure involves all these parameters as input - and still a proper matching mechanism has to be found. We 
address this problem as a decision making one. 



Multi-Criteria Decision Making Methodology 

According to Roy (1996) the general methodology of decision making problems includes four steps: (i) 
defining the object of the decision, that is the set of potential actions, and the problematic of the decision; (ii) 
studying the parameters influencing decision and defining so a set of criteria; (iii) choosing an appropriate multi- 
criteria aggregation method; (iv) proceeding at the activity of decision aid. 

Step One: The Object of Decision 

The first step includes definition of the decision variables in a' form of a consideration set/4. In the case of 
educational content brokering, this set includes all the available learning objects, which will be evaluated by the 
decision maker (in our case, the brokering agent). What is called ‘the problematic of the decision’ is the definition of 
what kind of evaluation or choice does the decision maker want to make upon the different objects available in set/4; 
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in the case of broker agents recommending a learning object, the decision problematic is “selection of one”. That is 
the selection of one action from the consideration set of the form ?={ai, a 2 , 

Step Two: The Criteria 

The next step is deciding what will be the criteria upon which the learning object will be evaluated. This 
means defining a consistent family of criteria, assuming that these criteria are non-decreasing value functions, 
exhaustive and non-redundant. Each criterion is defined on A as it follows: 

gf : A [g,.. ,g* ] C 91/ a ^ g(a) e Si , where [g,. ,g.*] is the criterion evaluation scale, with the worst 

level of the ith criterion, the best level of the /th criterion, g. (a) the evaluation or performance of action a on 

the ith criterion and g{ci) the vector of performances of actions on the n criteria (Jacquet-Lagreze & Siskos, 2001). 

In the case of educational content brokering, there are two ways to deal with the selection of the criteria. 
Recommendation is usually based on criteria as price, time of delivery, form of delivery, etc. - a common practice in e- 
commerce systems. We would like to introduce another aspect of recommendation in agent-based learning 
environments: based on the pedagogical profile of the learner. That is using the categories of the learner model as 
criteria; each learning object will be evaluated on how suitable it is for each category of learners. This is exactly the 
same as making an expression as “ TypeOfContent=’That’ ”. To be more specific, in a similar case an expert system 
would need a definition as: TypeOfContent="SuitableForActivists". In the case multiple criteria are used for 
the description of the content, this definition is transformed (remaining fully equivalent) to something similar to this: 

• SuitabilityOfContentForTheorists= "NotSuitableAtAll" ; 

• SuitabilityOfContentForActivists= "Perfectly suitable"; 

• SuitabilityOfContentForReflectors= "NotSuitableAtAll"; 

• SuitabilityOfContentForPragmatists= "NotSuitableAtAll"; 

Therefore, as criteria we will use the four categories of the Honey & Mumford, which can take their values from a 
5“Scaled climax of qualitative descriptions [“Not suitable at all”, “Not very suitable”, “Moderately suitable”, “Very 
suitable”, “Perfectly suitable”] showing the evaluation of each learning object upon each categories. It is obvious 
that a learning object (e.g. an on-line course) is never “Perfectly Suitable” for a learner category and “Not Suitable At 
All” for the rest -it rather addresses some of the needs of other categories of learners too. Therefore, such an 
evaluation provides the expert with a way to precisely describe the pedagogical ‘profile’ of a learning object. This 
meta-data description is a criteria definition that fully complies with the definition of the criteria according to Roy 
(1996). 

Step Three: The Utility Model 

First of all, we have to define the preference model that will be used from the broker agent to make the 
decision; as we have already stated, in our case we are using the Honey & Mumford learner model as a preference 
model. The expression of the model is then pr\Pi > P 2 > Pi > P 4 \ where pi, p 2 , P 3 , andp 4 are the weights inferred after 
the learner has completed the Honey & Mumford LSQ and the results have been processed. 

This is the point where the analyst has to define which evaluation method will be used; that is, which multi- 
criteria decision making method will be engaged in order for the decision to be best modeled and simulated. In this 
paper we will use one of the most traditional approaches, that leads to a functional representation g that can be 
formed directly from the criteria gj,...g 4 that constitute A. The goal of using this approach is to present how the 
methodology works - depending on the application, other more complex methods can be used. Thus, the 
comprehensive preference model is characterized by a unique synthesizing criterion g.* g(a)=V\gj(a),...,g„(a)], where 
V is an aggregation function. The function will be in the form: 

Ha) = '^p.g.(a) 

where g/a) are the evaluations of each learning object a regarding the suitability for each category of learners. The 
weights Pi consist the preference model, which are the parameters calculated by the Honey and Mumford LSQ. The 
final value of V is the total utility of each learning object for the learner under study. 




Step Four: The Activity of Decision Aid 

Let us now introduce the way the activity of decision making will be carried out, in the context of an agent- 
based system We define a generic architecture, with agents capable of handling and communicating descriptions of 
learning objects, as defined in the previous paragraph. The three basic roles are: 

The Assistant. The assistant agents are responsible for user needs and requirements elicitation, and formulation of 
requests into messages understandable by the broker repository system. The L Assistant is the agent that interacts 
with the user and can identify the request for a learning object. The CPAssistant is the agent that gets the learning 
object descriptions from the Content Providers, and publishes them as offers in the e-market. The TAssistant serves 
as the pedagogical counselor of the learner: it creates the learner model of the user and provides this model to the 
LBroker. The TAssistant can also be the expert that describes a learning object using the cognitive styles; this is a 
task that can be directly carried out by the Content Author too, so we will not focus more on this point. 

The Broker. The broker agents represent each human user in the e-market and facilitate educational content seeking 
or advertising. The Broker agents interact among each other in the Brokerage Pool and either advertise published 
learning objects, or make requests when looking for them. The LBroker evaluates the available offers from the 
CPBrokers in the way described in the previous paragraphs. The LBroker then recommends the content selection to 
the LAssistant. The Brokers are also responsible for negotiation among the participating parties upon the terms of 
the service provisioning (beyond the scope of this paper). 

The Matchmaker. This facilitator agent provides mediating services, by informing agents about other agents of the 
broker repository and their availability. The Matchmaker provides all necessary administrative support information to 
the human administrator of the agent pool. 

We can see that the Broker agent engaging the multi-criteria methodology previously described is the 
LBroker. In order to pedagogically evaluate the available learning objects and recommend one to the learner, the 
LBroker needs the learner model (provided by the user) and the meta-description of the available learning objects 
(provided by the CPBroker or the Tutor). Other kinds of evaluation of the offers (e.g. based on the price, time, and 
other terms of delivery) are beyond the scope of this paper. 



Usage Scenario 

Let us demonstrate how the recommendation procedure works in this agent-based system. A content author 
creates different e-learning courses concerning the same subject. The author wishes those courses to address 
different learner needs, depending on the cognitive style of the learner. When a learning object (course) is created, 
the content author also provides an evaluation of its suitability for each different category of the Honey & Mumford 
model. The author creates thus five different learning objects, with different pedagogical descriptions (as depicted in 
Table 1). 





LOl 


L02 


L03 


L04 


LOS 


Activist 


3 


2 


5 


3 


5 


Reflector 


1 


3 


1 


2 


2 


Theorist 


4 


5 


2 


1 


1 


Pragmatist 


3 


2 


1 


5 


2 



Table 1: The content author provides an description of each learning object (LO) on a scale from ‘1- Not Suitable At 
Air to ‘5-Perfectly Suitable’, regarding each one of the cognitive categories. 
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Activist 


Reflector 


Theorist 


Pragmatist 


LSQ values 


90% 


55% 


25% 


85% 


Normalized 


0,35 


0,22 


0,10 


0,33 



Table 2: The cognitive styles preference model, as derived from the Honey and Mumford Learning Styles 
Questionnaire (values are properly normalized in order to provide a sum of 1). 





LOl 


L02 


L03 


L04 


LOS 


Utility 


2,67 


2,51 


2,51 


3,25 


2,96 



Table 3: The total utilities of the available learning objects (a score upon the ’1-Not Suitable At All’ to ’5- Perfectly 
Suitable’ climax). 

Introducing the multi-criteria methodology presented in the previous section the Broker will select a learning object 
according to these steps: 

1. The Tutor provides a multi-attribute cognitive model of the user, according to the results of the Honey and 
Mumford model (Table 2). 

2. When the Learner’s Broker receives the five offers for learning objects, it calculates the total utility (that is 
suitability) of each one, using the methodology described in the previous section. 

3. The Learner’s Broker selects the most appropriate learning object to be presented to the user (Table 3). 

It is interesting to note in this example the difference between the application of a simple rule -based selection of the 
learning object (in this case the agent would classify the learner as ‘Activist’ and would propose the learning object 
L03) and the proposed methodology. Using the multi-attribute cognitive model and the multi-criteria evaluation 
model, we observe that the most appropriate learning object seems to be L04, being the only one suitable for the 
combination of categories that characterize the specific learner’s cognitive style. 



Conclusions and Future Work 



In this paper we presented an agent-based recommendation system for educational content brokering that 
implements a multi-criteria decision making methodology. Due to the differences of learners’ cognitive models, 
recommendation can be either carried out by classifying learners into broader categories that then are directly linked 
to learning objects ^.g. with rules like IF ‘thistypeofleamer’ THEN ‘thistypeofmaterial’) or by enhancing the 
mediating agent with the appropriate intelligence to propose the most suitable learning object available at the market. 
By engaging a multi-attribute utility (MAUT) model, the brokering agents do not need to posses intelligence in the 
form of a knowledge base or hard-coded rules but only to include the multi-attribute evaluation logic in their decision 
making module. In this way learners are modeled in a multi-attribute way, instead of being simply classified into a 
general learning category, learning objects are described according to their educational scopes, brokering agents 
dynamically evaluate all available proposals of content, and select the learning objects most suitable for the specific 
learner’s profile, and finally the knowledge the broker agents have to contain in permanent storage is minimized since 
only the MAUT evaluation formula is needed; all other input is provided at run-time. 

In this paper, the decision problem was modeled using the Honey and Mumford LSQ parameters as the 
preference model, but we believe that the proposed methodology can be similarly applied in the case of other 
cognitive style models too. Future research will be focused on applying of several other LS models, either separately 
or combined. An issue to be also studied is the application of more complex multi-criteria decision making 
methodologies for modeling the user preferences; such methodologies provide easier and more realistic means of 
expressing preferences, but require more complicated MCDM methods to conclude to the best proposal for each 
learner, as one of the famous family of the ELECTRE methods (Roy, 1996). 

Current international standardization efforts regarding syntax and ^mantics of a learner model that 
characterize a learner and his or her knowledge/abilities; i.e. IMS LIP (IMS, 2001), IEEE LTSC PAPI (IEEE, 2000) do 
not explicitly define information about the cognitive style of the learner. However, both the IMS LIP and IEEE PAPI 
Learner preference/performance data types provide data elements that allow cognitive style related information to be 
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stored in an appropriate form. These data types could be used in order to represent a multi-attribute representation of 
the cognitive style model of the learner, by an appropriate extension of the corresponding XML schema. 

The 1ST Project KOD “Knowledge-On -Demand” (KOD, 2001) engages a multi-agent brokering system for 
selection of learning objects and dynamic synthesis of on-line courses. In the case of KOD, intelligence is included 
into the learning objects in the form of rules that express the suitability of the object for each type of learners, so that 
courses can be constructed based on how the learning objects are characterized. This is a case very close to the 
problem we introduced, and we are currently studying ways to introduce the multi-criteria descriptions in the learning 
objects of KOD, in order for the KOD agents to be able to dynamically assess and synthesize courses based on the 
introduced decision making methodology and not on prescribed rules. 
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